Combining Multi-Scale Character Recognition and Linguistic Knowledge for Natural Scene Text OCR
Identifieur interne : 000074 ( France/Analysis ); précédent : 000073; suivant : 000075Combining Multi-Scale Character Recognition and Linguistic Knowledge for Natural Scene Text OCR
Auteurs : Khaoula Elagouni [France] ; Christophe Garcia [France] ; Franck Mamalet [France] ; Pascale Sébillot [France]Source :
English descriptors
- mix :
Abstract
Understanding text captured in real-world scenes is a challenging problem in the field of visual pattern recognition and continues to generate a significant interest in the OCR (Optical Character Recognition) community. This paper proposes a novel method to recognize scene texts avoiding the conventional character segmentation step. The idea is to scan the text image with multi-scale windows and apply a robust recognition model, relying on a neural classification approach, to every window in order to recognize valid characters and identify non valid ones. Recognition results are represented as a graph model in order to determine the best sequence of characters. Some linguistic knowledge is also incorporated to remove errors due to recognition confusions. The designed method is evaluated on the ICDAR 2003 database of scene text images and outperforms state-of-the-art approaches.
Url:
Affiliations:
- France
- Auvergne-Rhône-Alpes, Rhône-Alpes, Région Bretagne
- Lyon, Rennes
- Université Claude Bernard Lyon 1, Université de Lyon, Université de Rennes 1, Université européenne de Bretagne
Links toward previous steps (curation, corpus...)
- to stream Hal, to step Corpus: 000030
- to stream Hal, to step Curation: 000030
- to stream Hal, to step Checkpoint: 000066
- to stream Main, to step Merge: 000233
- to stream Main, to step Curation: 000229
- to stream Main, to step Exploration: 000229
- to stream France, to step Extraction: 000074
Links to Exploration step
Hal:hal-00753908Le document en format XML
<record><TEI><teiHeader><fileDesc><titleStmt><title xml:lang="en">Combining Multi-Scale Character Recognition and Linguistic Knowledge for Natural Scene Text OCR</title><author><name sortKey="Elagouni, Khaoula" sort="Elagouni, Khaoula" uniqKey="Elagouni K" first="Khaoula" last="Elagouni">Khaoula Elagouni</name><affiliation wicri:level="1"><hal:affiliation type="laboratory" xml:id="struct-121780" status="VALID"><orgName>Orange Labs R&D [Rennes]</orgName><desc><address><addrLine>4 rue du Clos Courtel, 35112 Cesson-Sévigné Cedex, France</addrLine><country key="FR"></country></address></desc><listRelation><relation active="#struct-300518" type="direct"></relation></listRelation><tutelles><tutelle active="#struct-300518" type="direct"><org type="institution" xml:id="struct-300518" status="VALID"><orgName>France Télécom</orgName><desc><address><country key="FR"></country></address></desc></org></tutelle></tutelles></hal:affiliation><country>France</country></affiliation></author><author><name sortKey="Garcia, Christophe" sort="Garcia, Christophe" uniqKey="Garcia C" first="Christophe" last="Garcia">Christophe Garcia</name><affiliation wicri:level="1"><hal:affiliation type="laboratory" xml:id="struct-2003" status="VALID"><orgName>Laboratoire d'InfoRmatique en Image et Systèmes d'information</orgName><orgName type="acronym">LIRIS</orgName><desc><address><addrLine>Bâtiment Blaise Pascal - 20, avenue Albert Einstein - 69621 Villeurbanne cedex</addrLine><country key="FR"></country></address><ref type="url">http://liris.cnrs.fr/</ref></desc><listRelation><relation active="#struct-33804" type="direct"></relation><relation active="#struct-126765" type="direct"></relation><relation active="#struct-194495" type="direct"></relation><relation name="- LYON" active="#struct-301232" type="direct"></relation><relation name="UMR5205" active="#struct-441569" type="direct"></relation></listRelation><tutelles><tutelle active="#struct-33804" type="direct"><org type="institution" xml:id="struct-33804" status="VALID"><orgName>Université Lumière - Lyon 2</orgName><orgName type="acronym">UL2</orgName><desc><address><addrLine>86, rue Pasteur - 69007 Lyon</addrLine><country key="FR"></country></address><ref type="url">http://www.univ-lyon2.fr</ref></desc></org></tutelle><tutelle active="#struct-126765" type="direct"><org type="institution" xml:id="struct-126765" status="VALID"><orgName>École Centrale de Lyon</orgName><orgName type="acronym">ECL</orgName><desc><address><addrLine>36 avenue Guy de Collongue - 69134 Ecully cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.ec-lyon.fr</ref></desc></org></tutelle><tutelle active="#struct-194495" type="direct"><org type="institution" xml:id="struct-194495" status="VALID"><orgName>Université Claude Bernard Lyon 1</orgName><orgName type="acronym">UCBL</orgName><desc><address><addrLine>43, boulevard du 11 novembre 1918, 69622 Villeurbanne cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.univ-lyon1.fr/</ref></desc></org></tutelle><tutelle name="- LYON" active="#struct-301232" type="direct"><org type="institution" xml:id="struct-301232" status="VALID"><orgName>Institut National des Sciences Appliquées</orgName><orgName type="acronym">INSA</orgName><desc><address><country key="FR"></country></address></desc></org></tutelle><tutelle name="UMR5205" active="#struct-441569" type="direct"><org type="institution" xml:id="struct-441569" status="VALID"><idno type="IdRef">02636817X</idno><idno type="ISNI">0000000122597504</idno><orgName>Centre National de la Recherche Scientifique</orgName><orgName type="acronym">CNRS</orgName><date type="start">1939-10-19</date><desc><address><country key="FR"></country></address><ref type="url">http://www.cnrs.fr/</ref></desc></org></tutelle></tutelles></hal:affiliation><country>France</country><placeName><settlement type="city">Lyon</settlement><region type="region" nuts="2">Auvergne-Rhône-Alpes</region><region type="old region" nuts="2">Rhône-Alpes</region></placeName><orgName type="university">Université Claude Bernard Lyon 1</orgName><orgName type="institution" wicri:auto="newGroup">Université de Lyon</orgName></affiliation></author><author><name sortKey="Mamalet, Franck" sort="Mamalet, Franck" uniqKey="Mamalet F" first="Franck" last="Mamalet">Franck Mamalet</name><affiliation wicri:level="1"><hal:affiliation type="laboratory" xml:id="struct-121780" status="VALID"><orgName>Orange Labs R&D [Rennes]</orgName><desc><address><addrLine>4 rue du Clos Courtel, 35112 Cesson-Sévigné Cedex, France</addrLine><country key="FR"></country></address></desc><listRelation><relation active="#struct-300518" type="direct"></relation></listRelation><tutelles><tutelle active="#struct-300518" type="direct"><org type="institution" xml:id="struct-300518" status="VALID"><orgName>France Télécom</orgName><desc><address><country key="FR"></country></address></desc></org></tutelle></tutelles></hal:affiliation><country>France</country></affiliation></author><author><name sortKey="Sebillot, Pascale" sort="Sebillot, Pascale" uniqKey="Sebillot P" first="Pascale" last="Sébillot">Pascale Sébillot</name><affiliation wicri:level="1"><hal:affiliation type="researchteam" xml:id="struct-2538" status="OLD"><idno type="RNSR">200218363F</idno><orgName>Multimedia content-based indexing</orgName><orgName type="acronym">TEXMEX</orgName><date type="start">2002-11-01</date><date type="end">2014-12-31</date><desc><address><addrLine>Campus de Beaulieu, 35042 Rennes cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.inria.fr/equipes/texmex</ref></desc><listRelation><relation active="#struct-419153" type="direct"></relation><relation active="#struct-300009" type="indirect"></relation><relation active="#struct-2494" type="direct"></relation><relation active="#struct-105160" type="indirect"></relation><relation active="#struct-117606" type="indirect"></relation><relation name="UMR6074" active="#struct-441569" type="indirect"></relation></listRelation><tutelles><tutelle active="#struct-419153" type="direct"><org type="laboratory" xml:id="struct-419153" status="VALID"><idno type="RNSR">198018249C</idno><orgName>Inria Rennes – Bretagne Atlantique </orgName><desc><address><addrLine>Campus de beaulieu35042 Rennes cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.inria.fr/centre/rennes</ref></desc><listRelation><relation active="#struct-300009" type="direct"></relation></listRelation></org></tutelle><tutelle active="#struct-300009" type="indirect"><org type="institution" xml:id="struct-300009" status="VALID"><orgName>Institut National de Recherche en Informatique et en Automatique</orgName><orgName type="acronym">Inria</orgName><desc><address><addrLine>Domaine de VoluceauRocquencourt - BP 10578153 Le Chesnay Cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.inria.fr/en/</ref></desc></org></tutelle><tutelle active="#struct-2494" type="direct"><org type="laboratory" xml:id="struct-2494" status="OLD"><orgName>Institut de Recherche en Informatique et Systèmes Aléatoires</orgName><orgName type="acronym">IRISA</orgName><desc><address><addrLine>Campus universitaire de Beaulieu - 35042 Rennes</addrLine><country key="FR"></country></address><ref type="url">http://www.irisa.fr/</ref></desc><listRelation><relation active="#struct-105160" type="direct"></relation><relation active="#struct-117606" type="direct"></relation><relation active="#struct-300009" type="direct"></relation><relation name="UMR6074" active="#struct-441569" type="direct"></relation></listRelation></org></tutelle><tutelle active="#struct-105160" type="indirect"><org type="institution" xml:id="struct-105160" status="VALID"><orgName>Université de Rennes 1</orgName><orgName type="acronym">UR1</orgName><desc><address><addrLine>2 rue du Thabor - CS 46510 - 35065 Rennes cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.univ-rennes1.fr/</ref></desc></org></tutelle><tutelle active="#struct-117606" type="indirect"><org type="institution" xml:id="struct-117606" status="VALID"><orgName>Institut National des Sciences Appliquées - Rennes</orgName><orgName type="acronym">INSA Rennes</orgName><desc><address><addrLine>20, avenue des Buttes de Coësmes - CS 70839 - 35708 Rennes cedex 7</addrLine><country key="FR"></country></address><ref type="url">http://www.insa-rennes.fr/</ref></desc></org></tutelle><tutelle name="UMR6074" active="#struct-441569" type="indirect"><org type="institution" xml:id="struct-441569" status="VALID"><idno type="IdRef">02636817X</idno><idno type="ISNI">0000000122597504</idno><orgName>Centre National de la Recherche Scientifique</orgName><orgName type="acronym">CNRS</orgName><date type="start">1939-10-19</date><desc><address><country key="FR"></country></address><ref type="url">http://www.cnrs.fr/</ref></desc></org></tutelle></tutelles></hal:affiliation><country>France</country><placeName><settlement type="city">Rennes</settlement><region type="region" nuts="2">Région Bretagne</region></placeName><orgName type="university">Université de Rennes 1</orgName><orgName type="institution" wicri:auto="newGroup">Université européenne de Bretagne</orgName></affiliation></author></titleStmt><publicationStmt><idno type="wicri:source">HAL</idno><idno type="RBID">Hal:hal-00753908</idno><idno type="halId">hal-00753908</idno><idno type="halUri">https://hal.archives-ouvertes.fr/hal-00753908</idno><idno type="url">https://hal.archives-ouvertes.fr/hal-00753908</idno><date when="2012-03-27">2012-03-27</date><idno type="wicri:Area/Hal/Corpus">000030</idno><idno type="wicri:Area/Hal/Curation">000030</idno><idno type="wicri:Area/Hal/Checkpoint">000066</idno><idno type="wicri:Area/Main/Merge">000233</idno><idno type="wicri:Area/Main/Curation">000229</idno><idno type="wicri:Area/Main/Exploration">000229</idno><idno type="wicri:Area/France/Extraction">000074</idno></publicationStmt><sourceDesc><biblStruct><analytic><title xml:lang="en">Combining Multi-Scale Character Recognition and Linguistic Knowledge for Natural Scene Text OCR</title><author><name sortKey="Elagouni, Khaoula" sort="Elagouni, Khaoula" uniqKey="Elagouni K" first="Khaoula" last="Elagouni">Khaoula Elagouni</name><affiliation wicri:level="1"><hal:affiliation type="laboratory" xml:id="struct-121780" status="VALID"><orgName>Orange Labs R&D [Rennes]</orgName><desc><address><addrLine>4 rue du Clos Courtel, 35112 Cesson-Sévigné Cedex, France</addrLine><country key="FR"></country></address></desc><listRelation><relation active="#struct-300518" type="direct"></relation></listRelation><tutelles><tutelle active="#struct-300518" type="direct"><org type="institution" xml:id="struct-300518" status="VALID"><orgName>France Télécom</orgName><desc><address><country key="FR"></country></address></desc></org></tutelle></tutelles></hal:affiliation><country>France</country></affiliation></author><author><name sortKey="Garcia, Christophe" sort="Garcia, Christophe" uniqKey="Garcia C" first="Christophe" last="Garcia">Christophe Garcia</name><affiliation wicri:level="1"><hal:affiliation type="laboratory" xml:id="struct-2003" status="VALID"><orgName>Laboratoire d'InfoRmatique en Image et Systèmes d'information</orgName><orgName type="acronym">LIRIS</orgName><desc><address><addrLine>Bâtiment Blaise Pascal - 20, avenue Albert Einstein - 69621 Villeurbanne cedex</addrLine><country key="FR"></country></address><ref type="url">http://liris.cnrs.fr/</ref></desc><listRelation><relation active="#struct-33804" type="direct"></relation><relation active="#struct-126765" type="direct"></relation><relation active="#struct-194495" type="direct"></relation><relation name="- LYON" active="#struct-301232" type="direct"></relation><relation name="UMR5205" active="#struct-441569" type="direct"></relation></listRelation><tutelles><tutelle active="#struct-33804" type="direct"><org type="institution" xml:id="struct-33804" status="VALID"><orgName>Université Lumière - Lyon 2</orgName><orgName type="acronym">UL2</orgName><desc><address><addrLine>86, rue Pasteur - 69007 Lyon</addrLine><country key="FR"></country></address><ref type="url">http://www.univ-lyon2.fr</ref></desc></org></tutelle><tutelle active="#struct-126765" type="direct"><org type="institution" xml:id="struct-126765" status="VALID"><orgName>École Centrale de Lyon</orgName><orgName type="acronym">ECL</orgName><desc><address><addrLine>36 avenue Guy de Collongue - 69134 Ecully cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.ec-lyon.fr</ref></desc></org></tutelle><tutelle active="#struct-194495" type="direct"><org type="institution" xml:id="struct-194495" status="VALID"><orgName>Université Claude Bernard Lyon 1</orgName><orgName type="acronym">UCBL</orgName><desc><address><addrLine>43, boulevard du 11 novembre 1918, 69622 Villeurbanne cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.univ-lyon1.fr/</ref></desc></org></tutelle><tutelle name="- LYON" active="#struct-301232" type="direct"><org type="institution" xml:id="struct-301232" status="VALID"><orgName>Institut National des Sciences Appliquées</orgName><orgName type="acronym">INSA</orgName><desc><address><country key="FR"></country></address></desc></org></tutelle><tutelle name="UMR5205" active="#struct-441569" type="direct"><org type="institution" xml:id="struct-441569" status="VALID"><idno type="IdRef">02636817X</idno><idno type="ISNI">0000000122597504</idno><orgName>Centre National de la Recherche Scientifique</orgName><orgName type="acronym">CNRS</orgName><date type="start">1939-10-19</date><desc><address><country key="FR"></country></address><ref type="url">http://www.cnrs.fr/</ref></desc></org></tutelle></tutelles></hal:affiliation><country>France</country><placeName><settlement type="city">Lyon</settlement><region type="region" nuts="2">Auvergne-Rhône-Alpes</region><region type="old region" nuts="2">Rhône-Alpes</region></placeName><orgName type="university">Université Claude Bernard Lyon 1</orgName><orgName type="institution" wicri:auto="newGroup">Université de Lyon</orgName></affiliation></author><author><name sortKey="Mamalet, Franck" sort="Mamalet, Franck" uniqKey="Mamalet F" first="Franck" last="Mamalet">Franck Mamalet</name><affiliation wicri:level="1"><hal:affiliation type="laboratory" xml:id="struct-121780" status="VALID"><orgName>Orange Labs R&D [Rennes]</orgName><desc><address><addrLine>4 rue du Clos Courtel, 35112 Cesson-Sévigné Cedex, France</addrLine><country key="FR"></country></address></desc><listRelation><relation active="#struct-300518" type="direct"></relation></listRelation><tutelles><tutelle active="#struct-300518" type="direct"><org type="institution" xml:id="struct-300518" status="VALID"><orgName>France Télécom</orgName><desc><address><country key="FR"></country></address></desc></org></tutelle></tutelles></hal:affiliation><country>France</country></affiliation></author><author><name sortKey="Sebillot, Pascale" sort="Sebillot, Pascale" uniqKey="Sebillot P" first="Pascale" last="Sébillot">Pascale Sébillot</name><affiliation wicri:level="1"><hal:affiliation type="researchteam" xml:id="struct-2538" status="OLD"><idno type="RNSR">200218363F</idno><orgName>Multimedia content-based indexing</orgName><orgName type="acronym">TEXMEX</orgName><date type="start">2002-11-01</date><date type="end">2014-12-31</date><desc><address><addrLine>Campus de Beaulieu, 35042 Rennes cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.inria.fr/equipes/texmex</ref></desc><listRelation><relation active="#struct-419153" type="direct"></relation><relation active="#struct-300009" type="indirect"></relation><relation active="#struct-2494" type="direct"></relation><relation active="#struct-105160" type="indirect"></relation><relation active="#struct-117606" type="indirect"></relation><relation name="UMR6074" active="#struct-441569" type="indirect"></relation></listRelation><tutelles><tutelle active="#struct-419153" type="direct"><org type="laboratory" xml:id="struct-419153" status="VALID"><idno type="RNSR">198018249C</idno><orgName>Inria Rennes – Bretagne Atlantique </orgName><desc><address><addrLine>Campus de beaulieu35042 Rennes cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.inria.fr/centre/rennes</ref></desc><listRelation><relation active="#struct-300009" type="direct"></relation></listRelation></org></tutelle><tutelle active="#struct-300009" type="indirect"><org type="institution" xml:id="struct-300009" status="VALID"><orgName>Institut National de Recherche en Informatique et en Automatique</orgName><orgName type="acronym">Inria</orgName><desc><address><addrLine>Domaine de VoluceauRocquencourt - BP 10578153 Le Chesnay Cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.inria.fr/en/</ref></desc></org></tutelle><tutelle active="#struct-2494" type="direct"><org type="laboratory" xml:id="struct-2494" status="OLD"><orgName>Institut de Recherche en Informatique et Systèmes Aléatoires</orgName><orgName type="acronym">IRISA</orgName><desc><address><addrLine>Campus universitaire de Beaulieu - 35042 Rennes</addrLine><country key="FR"></country></address><ref type="url">http://www.irisa.fr/</ref></desc><listRelation><relation active="#struct-105160" type="direct"></relation><relation active="#struct-117606" type="direct"></relation><relation active="#struct-300009" type="direct"></relation><relation name="UMR6074" active="#struct-441569" type="direct"></relation></listRelation></org></tutelle><tutelle active="#struct-105160" type="indirect"><org type="institution" xml:id="struct-105160" status="VALID"><orgName>Université de Rennes 1</orgName><orgName type="acronym">UR1</orgName><desc><address><addrLine>2 rue du Thabor - CS 46510 - 35065 Rennes cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.univ-rennes1.fr/</ref></desc></org></tutelle><tutelle active="#struct-117606" type="indirect"><org type="institution" xml:id="struct-117606" status="VALID"><orgName>Institut National des Sciences Appliquées - Rennes</orgName><orgName type="acronym">INSA Rennes</orgName><desc><address><addrLine>20, avenue des Buttes de Coësmes - CS 70839 - 35708 Rennes cedex 7</addrLine><country key="FR"></country></address><ref type="url">http://www.insa-rennes.fr/</ref></desc></org></tutelle><tutelle name="UMR6074" active="#struct-441569" type="indirect"><org type="institution" xml:id="struct-441569" status="VALID"><idno type="IdRef">02636817X</idno><idno type="ISNI">0000000122597504</idno><orgName>Centre National de la Recherche Scientifique</orgName><orgName type="acronym">CNRS</orgName><date type="start">1939-10-19</date><desc><address><country key="FR"></country></address><ref type="url">http://www.cnrs.fr/</ref></desc></org></tutelle></tutelles></hal:affiliation><country>France</country><placeName><settlement type="city">Rennes</settlement><region type="region" nuts="2">Région Bretagne</region></placeName><orgName type="university">Université de Rennes 1</orgName><orgName type="institution" wicri:auto="newGroup">Université européenne de Bretagne</orgName></affiliation></author></analytic></biblStruct></sourceDesc></fileDesc><profileDesc><textClass><keywords scheme="mix" xml:lang="en"><term>convolutional neural networks</term><term>language model</term><term>multi-scale character recognition</term><term>scene text recognition</term></keywords></textClass></profileDesc></teiHeader><front><div type="abstract" xml:lang="en">Understanding text captured in real-world scenes is a challenging problem in the field of visual pattern recognition and continues to generate a significant interest in the OCR (Optical Character Recognition) community. This paper proposes a novel method to recognize scene texts avoiding the conventional character segmentation step. The idea is to scan the text image with multi-scale windows and apply a robust recognition model, relying on a neural classification approach, to every window in order to recognize valid characters and identify non valid ones. Recognition results are represented as a graph model in order to determine the best sequence of characters. Some linguistic knowledge is also incorporated to remove errors due to recognition confusions. The designed method is evaluated on the ICDAR 2003 database of scene text images and outperforms state-of-the-art approaches.</div></front></TEI><affiliations><list><country><li>France</li></country><region><li>Auvergne-Rhône-Alpes</li><li>Rhône-Alpes</li><li>Région Bretagne</li></region><settlement><li>Lyon</li><li>Rennes</li></settlement><orgName><li>Université Claude Bernard Lyon 1</li><li>Université de Lyon</li><li>Université de Rennes 1</li><li>Université européenne de Bretagne</li></orgName></list><tree><country name="France"><noRegion><name sortKey="Elagouni, Khaoula" sort="Elagouni, Khaoula" uniqKey="Elagouni K" first="Khaoula" last="Elagouni">Khaoula Elagouni</name></noRegion><name sortKey="Garcia, Christophe" sort="Garcia, Christophe" uniqKey="Garcia C" first="Christophe" last="Garcia">Christophe Garcia</name><name sortKey="Mamalet, Franck" sort="Mamalet, Franck" uniqKey="Mamalet F" first="Franck" last="Mamalet">Franck Mamalet</name><name sortKey="Sebillot, Pascale" sort="Sebillot, Pascale" uniqKey="Sebillot P" first="Pascale" last="Sébillot">Pascale Sébillot</name></country></tree></affiliations></record>Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Ticri/CIDE/explor/OcrV1/Data/France/Analysis
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 000074 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/France/Analysis/biblio.hfd -nk 000074 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien
|wiki= Ticri/CIDE
|area= OcrV1
|flux= France
|étape= Analysis
|type= RBID
|clé= Hal:hal-00753908
|texte= Combining Multi-Scale Character Recognition and Linguistic Knowledge for Natural Scene Text OCR
}}
| This area was generated with Dilib version V0.6.32. |  |